Augmented reality devices for hazardous contaminant testing

ABSTRACT

Aspects of the disclosure relate to augmented reality devices for generating a composite scene including a real-world test environment and an augmented reality overlay visually representing a test area of the environment. Some devices can include wearable displays for displaying an overlay to a user, and other devices can include projectors for illuminating the test environment with the overlay.

CROSS-REFERENCE TO RELATED APPLICATIONS

This application claims the benefit of U.S. Provisional PatentApplication No. 62/561,541, filed on Sep. 21, 2017, entitled “AUGMENTEDREALITY DEVICES FOR HAZARDOUS CONTAMINANT TESTING,” the contents ofwhich are hereby incorporated by reference herein.

TECHNICAL FIELD

Systems and methods disclosed herein are directed to environmentalcontaminant testing, and, more particularly, to devices for accuratelymeasuring features of a sampled area, including physical dimensions ofthe sampled area.

BACKGROUND

Antineoplastic drugs are used to treat cancer, and are most often foundin a small molecule (like fluoruracil) or antibody format (likeRituximab). Detection of antineoplastic drugs is critical fordetermining if there is contamination or leakage where the drugs areused and/or dispensed, such as hospital and pharmacy areas.

The nature of antineoplastic drugs make them harmful to healthy cellsand tissues as well as the cancerous cells. Precautions should be takento eliminate or reduce occupational exposure to antineoplastic drugs forhealthcare workers. Pharmacists who prepare these drugs and nurses whomay prepare and administer them are the two occupational groups who havethe highest potential exposure to antineoplastic agents. Additionally,physicians and operating room personnel may also be exposed through thetreatment of patients, as patients treated with antineoplastic drugs canexcrete these drugs. Hospital staff, such as shipping and receivingpersonnel, custodial workers, laundry workers and waste handlers, allhave the potential to be exposed to these drugs during the course oftheir work. The increased use of antineoplastic agents in veterinaryoncology also puts these workers at risk for exposure to these drugs.

SUMMARY

Antineoplastic drugs are antiproliferative. In some cases they affectthe process of cell division by damaging DNA and initiating apoptosis, aform of programmed cell death. While this can be desirable forpreventing development and spread of neoplastic (e.g., cancerous) cells,antineoplastic drugs can also affect rapidly dividing non-cancerouscells. As such, antineoplastic drugs can suppress healthy biologicalfunctions including bone marrow growth, healing, hair growth, andfertility, to name a few examples.

Studies have associated workplace exposures to antineoplastic drugs withhealth effects such as skin rashes, hair loss, infertility (temporaryand permanent), effects on reproduction and the developing fetus inpregnant women, increased genotoxic effects (e.g., destructive effectson genetic material that can cause mutations), hearing impairment andcancer. These health risks are influenced by the extent of the exposureand the potency and toxicity of the hazardous drug. Although thepotential therapeutic benefits of hazardous drugs may outweigh the risksof such side effects for ill patients, exposed health care workers riskthese same side effects with no therapeutic benefit. Further, it isknown that exposures to even very small concentrations of antineoplasticdrugs may be hazardous for workers who handle them or work near them,and for known carcinogenic agents there is no safe level of exposure.

Environmental sampling can be used to determine the level of workplacecontamination by antineoplastic agents. Sampling and decontamination ofcontaminated areas is complicated, however, by a lack of quick,inexpensive methods to first identify these areas and then determine thelevel of success of the decontamination. Although analytical methods areavailable for testing for the presence of antineoplastic drugs inenvironmental samples, these methods require shipment to outside labs,delaying the receipt of sampling results.

In one example sampling system suitable for use with the devices of thepresent disclosure, work surfaces can be tested for the presence ofantineoplastic agents in an environment. Results of the test can beprovided very quickly, at the site of testing, so that the operator ofthe test, other personnel in the area, and/or remote systems can bealerted to the presence and/or concentration of antineoplastic agentsvery close in time to the test event, in some cases within 1-2 minutes.Methods of testing include providing the surface with a buffer solutionand wiping the wetted surface with an absorbent swab, or by wiping thesurface with a swab pre-wetted with the buffer solution. The bufferfluid can have properties that assist in picking up contaminants fromthe surface. In some implementations, the buffer fluid can haveproperties that assist in releasing collected contaminants from swabmaterial. The collected contaminants can be mixed into a homogeneoussolution for testing. The buffer solution, together with any collectedcontaminants, can be expressed or extracted from the swab to form aliquid sample. This liquid sample can be analyzed for presence and/orquantity of specific antineoplastic agents. For example, the solutioncan be provided onto an assay (such as but not limited to a lateral flowassay) which is read by an assay reader device to identify presenceand/or a concentration of the contaminant in the liquid sample. Thereader device can alternatively identify the concentration of thecontaminant on the test area, for example delivering a result ofconcentration per area (e.g., ng/ft²).

The accuracy of testing for the presence and/or concentration of acontaminant in a fluid sample is highly dependent on various testfactors. Test results can provide a measurement in the form ofconcentration of contaminant in a tested environment, for examplecontaminant mass per square unit area. Accordingly, precision andaccuracy in measuring the sampled area can be an important factor toobtain an accurate test result. Accurately measuring a specific samplearea can involve demarcating a test area of the surface to be tested andthen sampling the entire demarked area. Existing sampling systemsrequire the test operator to measure out test area dimensions and placephysical markers, such as adhesive dots, to define a rectangular testarea. The test operator of such existing systems is then responsible forensuring that the entire area is swabbed before cleaning up the markers.This approach has a number of drawbacks including requiring a lengthysetup, being subject to measurement and marker placement errors, lackingany tracking of actual sampled area, and increasing the risk of exposureof the test operator to potential hazardous drug contamination throughplacement and removal of the markers. For example, it increases theamount of time required to test an area to require the user to grab ameasuring tape, lay it down to measure the surface, and mark the area bypeeling off and placing round stickers. This can be cumbersome across avariety of test surfaces including counter tops, floors, walls, andothers. Not being able to accurate and consistent in selecting andmarking an area from which the user captures a sample causes theresulting test results to be inaccurate.

These and other problems are addressed in embodiments of the hazardousdrug collection and detection systems described herein, which includeaugmented reality devices configured to demarcate a test area and thentrack the portions of that area that are sampled by the test operator.With the disclosed systems, a user does not even have to demarcate anarea because this is done automatically by an augmented reality device,and the device can track the amount of area actually sampled both withinand outside of the pre-defined demarcated test area. Thus, the augmentedreality systems described herein both increase test result accuracy anddecrease time required for testing as well as mitigate exposureopportunities where the user may contact a contaminated surface. Thepresent technology provides improved accuracy for identifyingantineoplastic drug concentrations, including trace amounts ofantineoplastic drugs, compared to existing systems. The disclosedaugmented reality devices can communicate information relating to atested area to a detection system that analyzes the sample acquired fromthe tested area. The detection system is capable of accurately detectingquantities of even trace amounts of antineoplastic agents and ofproviding results quickly (including immediately after collection).Advantageously, testing and detection can occur at the location of thecollection so that immediate, quantitative assessment of contaminationlevel can be determined without the delay required for laboratory sampleprocessing.

Accordingly, one aspect relates to an augmented reality system forguiding collection of hazardous contaminant samples, comprising an imagecapture device configured for capturing images of a sampling environmentincluding a test surface; a display configured to display an augmentedreality overlay over a view of the sampling environment, the augmentedreality overlay including a visual representation of a boundary of atest area of the test surface; at least one computer-readable memoryhaving stored thereon executable instructions; and one or moreprocessors in communication with the at least one computer-readablememory and configured to execute the instructions to cause the system todetermine a size of the test area, determine a location of the test arearelative to one or both of the system and the sampling environment,cause output via the display of the visual representation of theboundary of the test area, monitor interactions between a user and thetest area as the user swabs the test area to collect a sample, determinean actual area swabbed by the user based on the monitored interactions,the size of the test area, and the location of the test area, andtransmit an indication of the actual area swabbed.

In some embodiments of the system, the one or more processors areconfigured to execute the instructions to cause the system to analyzedata from the image capture device to monitor the interactions betweenthe user and the test area. In some embodiments of the system, the oneor more processors are configured to execute the instructions to causethe system to transmit the indication of the actual area swabbed to atest device identified for analysis of the sample. In some embodimentsof the system, the one or more processors are configured to execute theinstructions to cause the system to receive an indication of a level ofcontamination of the test surface and output an alert to the userindicating the presence of contamination. In some embodiments of thesystem, the one or more processors are configured to execute theinstructions to cause the system to receive an indication of a level ofcontamination of the test surface and output an alert to the userindicating the level of contamination.

In some embodiments of the system, the one or more processors areconfigured to execute the instructions to cause the system to modifypresentation of the test area in the augmented reality overlay to showswabbed areas of the test area using a first visual depiction and toshow unswabbed areas of the test area using a second visual depiction.In some further embodiments, the first visual depiction is differentthan the second visual depiction. In some further embodiments, the oneor more processors are configured to execute the instructions to causethe system to identify the swabbed areas based on the monitoredinteractions between the user and the test area. In some furtherembodiments, the one or more processors are configured to execute theinstructions to cause the system to display a trail over the swabbedareas.

In some embodiments of the system, the one or more processors areconfigured to execute the instructions to cause the system to maintain,in the augmented reality overlay, the location of the test area relativeto one or both of the system and the sampling environment as the usermoves around the sampling environment. In some embodiments of thesystem, one or more processors are configured to execute theinstructions to cause the system to compare the actual area swabbed bythe user to a predetermined threshold swabbed area, and in response todetermining that the actual area swabbed is equal to the predetermineddesired swabbed area, provide an indication to the user to terminateswabbing the test area. In some embodiments of the system, the sample isa liquid sample. In some embodiments of the system, the one or moreprocessors are configured to execute the instructions to cause thesystem to receive data from the image capture device representing a testdevice after provision of the sample to the test device, and analyze thereceived data to identify the presence of a hazardous contaminant and/ora level of contamination of a hazardous contaminant on the test surface.

Another aspect relates to an augmented reality apparatus for guidingcollection of hazardous contaminant samples, comprising an image capturedevice configured for capturing images of a test surface; a projectorconfigured to project an augmented reality overlay onto the testsurface, the augmented reality overlay including a visual representationof a boundary of a test area of the test surface; at least onecomputer-readable memory having stored thereon executable instructions;and one or more processors in communication with the at least onecomputer-readable memory and configured to execute the instructions tocause the system to cause output via the display of the visualrepresentation of the boundary of the test area, analyze data receivedfrom the image capture device to monitor interactions between a user andthe test area as the user swabs the test area to collect a sample,determine an actual area swabbed by the user based on the monitoredinteractions, and transmit an indication of the actual area swabbed.

In some embodiments of the apparatus, the sample is a liquid sample. Insome embodiments of the apparatus, the one or more processors areconfigured to execute the instructions to cause the system to receive anindication of a level of contamination of the test surface and output analert to the user indicating the level of contamination. In someembodiments of the apparatus, the one or more processors are configuredto execute the instructions to cause the system to receive an indicationof a level of contamination of the test surface and output an alert tothe user indicating the presence of contamination. In some embodimentsof the apparatus, the one or more processors are configured to executethe instructions to cause the system to transmit the indication of theactual area swabbed to a test device identified for analysis of thesample.

In some embodiments of the apparatus, the one or more processors areconfigured to execute the instructions to cause the system to modifypresentation of the test area in the augmented reality overlay toindicate swabbed areas of the test area. In some further embodiments,the one or more processors are configured to execute the instructions tocause the system to identify the swabbed areas based on the monitoredinteractions between the user and the test area. In some furtherembodiments, the one or more processors are configured to execute theinstructions to cause the system to modify presentation of the test areain the augmented reality overlay to show swabbed areas of the test areausing a first visual depiction and to show unswabbed areas of the testarea using a second visual depiction. In some further embodiments, theone or more processors are configured to execute the instructions tocause the system to stop projection of a pattern onto the swabbed areasand maintain projection of the pattern onto the unswabbed areas.

In some embodiments of the apparatus, the one or more processors areconfigured to execute the instructions to cause the system to comparethe actual area swabbed by the user to a predetermined swabbed area, andin response to determining that the actual area swabbed is equal to thepredetermined swabbed area, provide an indication to the user toterminate swabbing the test area.

Another aspect relates to a non-transitory computer-readable mediumstoring instructions that, when executed, cause a physical computingdevice to perform operations for guiding collection of a sample of ahazardous contaminant, the operations comprising causing output of avisual representation of a boundary of a test area for guiding a user tocollect the sample of the hazardous contaminant from the test area;monitoring interactions between the user and the test area as the userswabs the test area to collect the sample; determining an actual areaswabbed by the user based on the monitored interactions; identifying atest device designated for analysis of the sample; and transmitting, tothe test device, an indication of the actual area swabbed.

In some embodiments, the operations further comprise modifyingpresentation of the test area to indicate swabbed and unswabbed areas ofthe test area. In some embodiments, the operations further compriseidentifying the swabbed and unswabbed areas of the test area based onmonitoring the interactions. In some embodiments, causing output of thevisual representation of the boundary of the test area comprisesoverlaying the visual representation over a view the test area through atransparent near eye display. In some embodiments, causing output of thevisual representation of the boundary of the test area comprisesoverlaying the visual representation over an image of the test area toform a composite view, and displaying the composite view to the user. Insome embodiments, causing output of the visual representation of theboundary of the test area comprises projecting the visual representationonto the test area.

BRIEF DESCRIPTION OF THE DRAWINGS

The disclosed aspects will hereinafter be described in conjunction withthe appended drawings, provided to illustrate and not to limit thedisclosed aspects, wherein like designations denote like elements.

FIGS. 1A-1D graphically illustrate steps of an example method ofcollecting and testing a liquid sample as described herein.

FIG. 2 depicts an example augmented reality display of a test areasampling environment as described herein.

FIG. 3 depicts a high level schematic block diagram of an exampleaugmented reality device that can be used to generate and display theexample display of FIG. 2.

FIG. 4 illustrates an example process for implementing an augmentedreality test area sampling environment, for example the display of FIG.2.

FIGS. 5A and 5B depict an example an example augmented realityprojection onto a test area sampling environment as described herein.

FIG. 6 depicts a high level schematic block diagram of an exampleprojection device that can be used to generate and display the exampleprojections of FIGS. 5A and 5B.

FIG. 7 illustrates an example process for implementing a projected testarea sampling environment, for example the projections of FIGS. 5A and5B.

DETAILED DESCRIPTION

Embodiments of the disclosure relate to systems and techniques fordetection of hazardous environmental contaminants, such as but notlimited to antineoplastic drugs used in the treatment of cancer, withincreased sensitivity to trace concentrations of antineoplastic drugs incollected samples. A kit for such testing can include a collectionsystem and a testing device, and the collection system can include anaugmented reality device for demarcating the test area and tracking howmuch of the demarcated area is sampled. Throughout this disclosure,example systems, kits, and methods will be described with reference tocollection, testing, and detection of antineoplastic agents, but it willbe understood that the present technology can be used to collect, test,and detect any particle, molecule, or analyte of interest.

A precise method of demarcating and sampling from a specified area canbe important in order to obtain an accurate test result in the form ofdrug mass per square unit area (e.g., nanograms per square centimeter).For example, a sample can be collected from a test surface by using abuffer liquid to wet the surface and using a swab to absorb the bufferliquid and any particles of hazardous drug contamination. When thesample is tested, a test device may be able to identify theconcentration of the hazardous drug in the volume of the liquid sample.In order to convert this measurement into a measurement of drugconcentration on the test surface, some implementations can use thefollowing formula:α=(Cv _(b))/(Aη _(p)η_(e))where α represents the contamination surface density (e.g., ng/cm²), Crepresents the concentration of the sample in the liquid sample, v_(b)represents the fluid volume of the buffer solution used to collect thesample, A represents the surface area swabbed, η_(p) represents the pickup efficiency of the swab material and buffer solution, and η_(e)represents the extraction efficiency of contaminant picked up by theswab material. The goal is to have a high concentration signal with lowvariability, however noise (e.g., variation) in these variables cancause the test to generate either false positive or false negativeresults. The disclosed augmented reality systems provide guidance forreducing the variation in the surface area swabbed, leading toheightened accuracy in sample testing, and in particular to a moreaccurate contamination surface density measurement.

Embodiments of the systems and methods described herein canadvantageously determine two important aspects regarding contaminationof a tested surface quickly and with high precision. First, thedisclosed systems and methods can determine the presence of even a verysmall amount of a hazardous contaminant. This provides an importantbenefit over manual sampling (e.g., sampling performed without thedisclosed augmented reality overlays and area tracking), because ifthere are just a few molecules on the surface, the user may miss themolecules entirely if they do not sample the test area in a regular,constrained, precise way. This type of sampling can lead to a falsenegative, leading to a missed opportunity to fix a leak or breach ofprotocol. In one example, the false negative reading may lead tohealthcare workers continuing work in the tested area, resulting intheir exposure to the hazardous contaminant. The disclosed augmentedreality devices ensure the user is reliably informed of the presence ofeven small amounts of hazardous agent, for example by guiding the userto perform a thorough sampling and by tracking actual sampled area.Second, the disclosed systems and methods can be used to more preciselydetermine the concentration of a detected hazardous contaminant byproviding an accurate metric regarding actual sampled area. This isimportant because the presence of a very small or trace concentrationsof certain hazardous drugs may be tolerable or even expected within anenvironment in some scenarios, but the difference between a smaller,acceptable trace concentration and a larger, unacceptable andpotentially dangerous trace concentration may be very small (e.g., onthe order of nanograms per centimeter). The disclosed augmented realitydevices enable the user to now know very quickly and reliably if theconcentration of a hazardous contaminant has elevated to dangerousconditions. Further, advantages of the systems and methods disclosedherein are not limited to guiding a user that is swabbing a test surfacein order to heighten accuracy of the test result. The augmented realitydevices advantageously minimize the spread of contamination by providingthe user with test area demarcation without requiring the user tocontact the test surface, and by guiding the user to collect a sample ina defined, highly constrained process. The sample collection guidancecan minimize the spread of existing contamination by helping to reduceunintended spillage and uncontrolled spread of buffer solution,unintended spreading of antineoplastic agent to other surfaces that arenot contaminated, and unintended spreading of antineoplastic agent tothe user.

As used herein, “augmented reality” refers to a live direct view orindirect view of a physical, real-world environment having elementsaugmented by a computer-generated visual overlay, for example images,projected shapes or patterns, user-interface elements, and the like. Alive direct view refers to the user looking directly at the environment,for example through a transparent display screen or at an environmentoverlaid with a projection, while an indirect view refers to the userviewing an image of the environment. Certain elements in an augmentedreality environment may be interactive and digitally manipulable throughuser input or feedback to the augmented reality device, for examplethrough automated gesture recognition, spoken commands, and/or userinteraction with physical controls (e.g., buttons, joysticks,touch-sensitive panels, etc.) of the device.

An augmented reality overlay as described herein can be presented inreal time. As used herein, “real time” describes computing systems thataugment real-world processes at a rate that substantially matches thatof the real process. In order to substantially match the rates, thedisclosed real time systems provide responses within specific timeconstraints, often in the order of milliseconds or microseconds. Assuch, the disclosed real time augmented reality systems can augment theenvironment of the user (or an image of the environment) with anaugmented reality overlay suitable for that environment as the user isstill experiencing that environment. From the perspective of the user, areal time system may present no perceptible lag in updating theaugmented reality overlay when changes occur in the real environment.

Although described primarily within the context of an augmented reality,it will be appreciated that the disclosed area demarcation and trackingtechniques can also be implemented in a virtual reality environment fortesting contamination of hazardous drugs, where the virtual realityenvironment permits user interaction with the real-world testingenvironment.

Drugs successfully treat many types of illnesses and injuries, butvirtually all drugs have side effects associated with their use. Not alladverse side effects classify as hazardous, however. In the presentdisclosure, the term “hazardous drugs” is used according to the meaningadopted by the American Society of Health-System Pharmacists (ASHP),which refers to a drug as hazardous if studies in animals or humans haveindicated that exposures to them have any one of four characteristics:genotoxicity; carcinogenicity; teratogenicity or fertility impairment;and serious organ damage or other toxic manifestation at low doses inexperimental animals or treated patients.

Although described in the example context of ascertaining theconcentration of hazardous drugs such as antineoplastic agents, it willbe appreciated that the disclosed devices and techniques for demarcatingand tracking a test sampling area can be used to detect the presenceand/or concentration of any analyte of interest. An analyte can include,for example, drugs (both hazardous and non-hazardous), antibodies,proteins, haptens, nucleic acids and amplicons.

Various embodiments will be described below in conjunction with thedrawings for purposes of illustration. It should be appreciated thatmany other implementations of the disclosed concepts are possible, andvarious advantages can be achieved with the disclosed implementations.

Overview of Example Sampling Method

FIGS. 1A-1D graphically illustrate steps of an example method ofcollecting and testing a liquid sample as described herein. FIG. 1Aillustrates example steps of a testing method 100A for testing for thepresence of an analyte on a test surface. One, some, or all of thedepicted blocks of FIG. 3A can be printed as graphical user interfaceinstructions on the packaging of an assay and/or collection kit, or canbe presented on a display screen of an assay reader device, a test areaterminal, or a personal computing device of the user.

At block 101, the user can identify a sample location and gather acollection kit, assay cartridges, and a template. The collection kit caninclude a swab attached to a handle and a collection container. In someexamples, the swab is pre-wetted with buffer solution and packagedtogether with the handle in a first sealed pouch and the collectioncontainer is packaged in a second sealed pouch. The assay cartridge mayinclude an assay device housed inside a cartridge having a window orport aligned with a sample receiving zone of the assay device. In oneimplementation, the assay device is a test strip, for example but notlimited to a lateral flow assay test strip. Also at block 101 the usercan put on clean gloves prior to each sample collection and/or openingof the collection kit, both to protect the user from potentialcontamination on the surface and to protect the collected sample fromcontamination on the user's hands.

At block 102, the user can establish a test area on the test surface.For example, the user can place a template (physical or projected) overthe intended location to clearly demarcate the area that will beswabbed. As described herein, block 102 can involve a user putting onand/or activating an augmented reality device to demarcate the testarea. Also at block 102 the user can open the collection kit packaging,including opening the separately-packaged swab and handle.

At block 103, the user can swab the entire test area with thepre-moistened swab. The user can swab the test area using slow and firmstrokes. As shown, the user can methodically pass the swab in straightlines along the height of the test area all the way across the width ofthe test area. The test area may be one square foot in some embodiments,for example demarcated as a 12 inches by 12 inches (144 square inches)region. Other examples can use greater or smaller areas for collectionincluding 10 inches by 10 inches, 8 inches by 8 inches, 6 inches by 6inches and 4 inches by 4 inches, non-square rectangular regions (e.g., a9 inches by 16 inches rectangle), and non-rectangular regions (e.g.circles). As described herein, the test area can be demarcated via anaugmented reality user interface, and the actual area sampled can betracked and automatically calculated by a device having a camerapositioned to observe the test area. The demarcation, tracking, and areacalculation can be performed by an augmented reality device as describedherein. The area that a user is instructed by the device to sampleduring a given test can be determined dynamically by the device, forexample based on the nature of the surface. For example, swabbing acountertop may use a default swab area of a 12 inches by 12 inchesregion, while the device may determine to use a smaller region forswabbing an IV pole, with this determination and the size of the smallerregion being based on determination of the size of the IV pole in imagescaptured by the device.

At block 104, the user can insert the swab into the collectioncontainer. In some examples, the collection container includes at-shaped well. Though not illustrated, the swab may have a t-shapedcross-section that substantially matches that of the container well. Theuser seals the container with a top that includes a dripper cap, andfully inverts (e.g., turn upside down and then return to right-side-up)the sealed container five times. During these inversions, the liquid inthe reservoir of the container washes primarily over the swab materialdue to the cross-sectional shape and other features of the reservoir,and the handle slides within the reservoir due to the reservoir having agreater height than the handle. As described herein, the inversioncombined with the geometries of the container and handle and the flow ofthe buffer solution can extract collected contaminants from the swabmaterial. In one non-limiting example, the user does not invert oragitate the container before moving to the next step.

At block 106, the user can leave the swab and handle inside thecontainer, remove the dripper cap, and squeeze (or allow gravity todraw) one or more drops (for example but not limited to four drops) intothe sample well on one or more assay cartridges. For example, in someembodiments the user may drop sample onto multiple assays each designedto test for a different drug. In some examples anywhere between threeand ten drops can produce suitable results on the assay. A drop is anapproximated unit of measure of volume corresponding to the amount ofliquid dispensed as one drop from a dropper or drip chamber viagravitational pull (sometimes aided by a positive pressure createdwithin the container holding the liquid). Though the precise volume ofany given drop depends upon factors such as the surface tension of theliquid of the drop, the strength of the gravitational field pulling onthe drop, and the device and technique used to produce the drop, it iscommonly considered to be a volume of 0.05 mL. In alternate embodimentsthe user may mechanically couple a fluid transfer portion of thecollection device to a fluid transfer portion of the assay device torelease a controlled volume of sample through a closed fluid pathway,for example as shown in FIG. 5C.

At block 107, the user can use a timer to allow the sample to developfor a period of time. For example, the sample can develop for about oneminute, about two minutes, about three minutes, about four minutes,about five minutes, about six minutes, or some other amount of time.Other development times are possible. In some embodiments the timer canbe built in to the programming of the reader device that reads theassay. The development time can vary depending on the particular testthat is being performed and the particular operating parameters of theassay device.

At block 108, the user can insert the assay cartridge into an assayreader device. The assay cartridge can be inserted into the ready deviceprior to or after the sample is developed, depending upon theoperational mode of the device. In some embodiments, the user maysequentially insert multiple cartridges for testing different aspects ofthe sample or for ensuring repeatability of test results.

At block 109, the assay reader device reads portions of the insertedcartridge (including, for example, detecting optical signals fromexposed areas of a capture zone of a test strip housed in thecartridge), analyzes the signals to determine optical changes to testzone location(s) and optionally control zone location(s), determines aresult based on the optical changes, and displays the result to theuser. The device can optionally store the result or transmit the resultover a network to a centralized data repository. As illustrated, thedevice displays a negative result for the presence of Doxorubicin in thesample. In other embodiments the device can display a specific detectedconcentration level in the sample and/or determined for the test area,and optionally can display confidence values in the determined result.

Embodiments of the reader devices described herein can determine thepresence or the absence of a hazardous drug on a tested surface with ahigh degree of confidence, and display an indication of this test resultto a user very quickly (in some instances, within 1 to 2 minutes) afterthe user tests the surface. In some cases, the reader device candetermine a concentration of contamination and display an indication ofthe determined concentration to the user very quickly (in someinstances, within 1 to 2 minutes) after the user tests the surface. Instill further examples, the reader device correlates a detected level ofcontamination with a risk of human uptake and/or risk of harmfulexposure to humans. To illustrate in one non-limiting example, anunintended human uptake of 1.0 ng/cm² of Cyclophosphamide, a hazardousantineoplastic drug, can be deemed a harmful exposure and/or exposure toa carcinogen. It will be understood that a different level ofcontamination of Cyclophosphamide could be established as a thresholdfor harmful exposure, and that the level of contamination for variousantineoplastic drugs can be set to different levels depending on theneeds of the user and the testing environment.

In this example, the reader device is configured to detect a level ofcontamination of Cyclophosphamide for a 12 inch by 12 inch (just as anexample) sampled area that is 1/10^(th) of this 1.0 ng/cm² thresholdlevel of Cyclophosphamide contamination, or 0.1 ng/cm². For example, thedynamic range of the assay test device (and reader devices describedherein that read the disclosed assay devices) can be capable ofdetecting a level of contamination of Cyclophosphamide as low as about0.1 ng/cm² per 12 inch by 12 inch sample test area. In one non-limitingembodiment, the reader device is configured to display an indication ofan actual measured concentration of Cyclophosphamide. For example, adisplay on the reader device may display the reading “0.085 ng/cm²” tothe user upon completion of reading the test device. In anothernon-limiting embodiment, the reader device is configured to indicate abinary result to the user based on an actual measured concentration ofCyclophosphamide. For example, a display on the reader device maydisplay the reading “−” or “− Cyclophosphamide” to the user uponcompletion of reading the test device when the actual measuredconcentration of Cyclophosphamide is less than about 0.1 ng/cm²(equivalent to a 93 ng mass of Cyclophosphamide for a 12 inch by 12 inchtest sample area). The display on the reader device may display thereading “+” or “+ Cyclophosphamide” to the user upon completion ofreading the test device when the actual measured concentration ofCyclophosphamide is about 0.1 ng/cm² or greater (equivalent to a 93 ngmass of Cyclophosphamide for a 12 inch by 12 inch test sample area).

In some examples, the reader device is configured to correlate an actualmeasurement of contamination with a risk of human uptake and/or risk ofharmful exposure to humans and to display an indication of the risk tothe user upon completion of reading the test device. For instance, thereader device may be configured to correlate an actual measuredconcentration of Cyclophosphamide of less than about 0.1 ng/cm² as areading within a window of acceptable error and/or with a low risk ofharmful exposure. In this case, the reader device can display a readingof “No further action” to the user. The reader device can be configuredto correlate an actual measured concentration of Cyclophosphamide ofabout 0.1 ng/cm² (equivalent to a 93 ng mass of Cyclophosphamide for a12 inch by 12 inch test sample area) with a moderate risk of harmfulexposure. In this case, the reader device can display a reading of“Notify others; Begin Decontamination” to the user. The reader devicecan be configured to correlate an actual measured concentration ofCyclophosphamide of greater than about 0.1 ng/cm² (equivalent to a 93 ngmass of Cyclophosphamide for a 12 inch by 12 inch test sample area) as areading within a window of unacceptably high contamination. In thiscase, the reader device can display a reading of “Evacuate immediately”to the user. The reader device may also automatically transmit a warningor alert to the user with a warning sound or light (for example, a voiceprompt or bright flashing light); transmit a warning or alert to otherpersonnel within a distance of the reader device and the tested surface(for example, initiate voice prompts to evacuate the immediate area,emit a high-decibel siren, etc.); and/or transmit a warning or alert topersonnel within or outside the physical location where the test eventoccurred (transmit, via a wired or wireless connection, an emergencynotification to a head pharmacist, nurse, manager, safety officer, orregulatory agency that includes location of the test event, hazardousdrug name, and the measured concentration of the hazardous drug). Theseexamples are not intended to be limiting and it will be understood thatother concentrations, thresholds, display readings, and warnings can beimplemented in the systems described herein.

After testing the user can re-seal the container with the dripper capand dispose of the collection device and assay (for example incompliance with hazardous waste regulations). Optionally, the user canexecute any needed decontamination procedures, re-test a decontaminatedsurface, and perform required reporting of the result.

FIG. 1B illustrates another testing method 100B that depicts details ofsteps 103, 104, and 106 of the process 100A using an alternateembodiment of the collection device.

The method 100B begins at step 105, in which a user can remove a handle140 from a container 130 containing a predetermined volume of bufferfluid 135. The handle 140 has a swab 245 secured to one end that ispre-wetted with the buffer fluid 135. In other implementations, the usercan separately apply a fluid that did not originate from the container130 to the test surface. For example, the buffer fluid 135 can beprovided separately, applied to the test surface, and absorbed using theswab 145. The buffer fluid 135 helps lift contaminants from the testsurface into the swab.

At step 110, optionally in some embodiments the swab head can rotate toassist in making and maintaining contact between the swab 145 and thetest surface 150. The handle 140 can have processing intelligence thatcommunicates with the augmented reality device as described herein, forexample by having a tracking beacon (e.g., reflector array, active radiofrequency, position sensor, etc.) that can be used during sampling toenable a more accurate assessment of actual swabbed area.

At step 115, the user can swab a designated test area of the testsurface 150. It can be preferable in some implementations to swab theentirety of the test area and only within the test area so as togenerate an accurate measurement of the concentration of thecontaminant, particularly for contaminants where even small quantitiesper area are harmful to users. The disclosed augmented reality devicescan be used to assist with demarcating and tracking the swabbed area.Swabbing the entirety of the test area and only within the test area canalso allow a reader device as described herein to generate an accuratemeasurement of the concentration of the contaminant per unit area insituations where a very small amount of contaminant is present. Even ifthe amount of contaminant detected is very small and not immediatelyharmful to persons in the immediate area, detection of contaminant inany amount can alert the user to a leak or unintended release ofhazardous material. Further, for some hazardous drugs there is no safeexposure level. As such, some embodiments of step 115 can involveactivating an augmented reality device to generate an area demarcationover the test area to assist the user with swabbing only a predeterminedarea, and can further involve monitoring the user's actions to determinethe actual sampled area and/or when total sampling of the demarcatedarea is complete.

At step 120, the user can replace the swab 145 and handle 140 into thecollection container 135. Optionally, the user and/or structure of thecontainer can agitate the swab to release collected contaminants intothe fluid within the container 135.

At step 125, the user can transfer fluid to a test device, such as butnot limited to a cartridge 155 containing a lateral flow assay includinga test strip. For example, the user can drip fluid from the container130 onto a sample receiving zone of the test strip. In some embodiments,the cartridge 155 (or other test system) and container 130 can bestructured to mechanically mate via a fluid-tight connection so as toprevent accidental exposure of potentially contaminated fluid to usersand/or the testing environment.

FIG. 1C illustrates a further step of inserting the cartridge 155 intoan aperture 170 of reader device 160. Though not illustrated, furthersteps can include operating the reader device 160 to detect a result ofthe test (for example, by imaging the test strip), analyze the testresult, and display results of the test. FIG. 1D illustrates the readerdevice 160 displaying a test result on display 180. In this case, thetest result indicates a concentration of the analyte of interest of 3ng/ml.

The device 160 can be an assay reader device having an aperture 170 forreceiving an assay test strip and cartridge 155 and positioning the teststrip so that the detection zones are positioned in the optical path ofimaging components located inside the device 160. The device can alsouse these or additional imaging components to image a bar code on thecartridge, for example to identify which imaging techniques and analysisto perform.

Some embodiments of the device 160 can be configured to perform aninitial scan using a bar code scanner to image one or more bar codes,for example provided on cartridges inserted into the aperture 170 or onseparate identifiers. A bar code can identify the type of test to beperformed, the person conducting the test, the location of the test,and/or the location in the facility of the test surface (for examplepharmacy, nursing area, cabinet #, bed #, chair #, pump #, etc.). Afterreading any bar code identifiers the cartridge 155 is then inserted intothe reader as shown in FIGS. 1A and 1C.

The device 160 can include a button 175 that readies the device for useand provides an input mechanism for a user to operate the device. Insome embodiments device operation mode can be set via a number orpattern of clicks of the single button 175 of the device 160. Forexample, in some implementations a single press of the button 175 canpower on the device 160 and set the device 160 to a default operationmode, and the device 160 can implement the default operation mode uponinsertion of a cartridge. A double-click of the button 175 can initiatean alternate operation mode that is different than the default operationmode. Other numbers or patterns of pressing the single button 175 by auser can provide instructions to the processor of the device regarding adesired operation mode. Embodiments of a device 160 are described hereinwith reference to a single button, but other features allowing a user toselect and switch between device operation modes are possible (such asbut not limited to a single switch, knob, lever, or handle).

One example of a device operation mode is end-point read mode. In theend-point read mode, the user prepares and incubates the assay outsideof the device 160 and tracks the development time of the assay. Forexample, an assay for determining Methotrexate or Doxorubicinconcentration can have a development time of 5 minutes, so the userwould apply the fluid to the assay from a collection device as describedherein and wait for 5 minutes. At the end of the 5 minutes the userwould insert the assay 155 into the device 160 to obtain a test result.Accordingly, when operating in end-point read mode the device 160 canprovide instructions, for example audibly or on a visual display, thatinstruct a user to wait for a predetermined time after applying a sampleto an assay before inserting the assay in the device 160. In otherembodiments, when operating in end-point read mode the device 160 maynot display any instructions but may simply read an assay upon insertioninto the device 160. Upon insertion of the assay into the base device160, an optical reader of the device can collect data (for example,image data) representing the assay for analysis in determining a resultof the assay. In some embodiments end-point read mode can be the defaultoperation mode of the device 160.

Another example of a device operation mode is walkaway mode. Whenoperating in walkaway mode, the device 160 can provide instructions forthe user to insert the assay immediately after or during application ofthe sample. In the walkaway mode according to one embodiment, the usercan apply the specimen to the assay and immediately insert the assayinto the device 160. The assay will develop inside the device 160 andthe device 160 can keep track of the time elapsed since insertion of theassay 155. At the end of the predetermined development time, the device160 can collect data (for example, image data) representing the assay.In implementations where the device 160 is an imaging reader, the device160 can analyze the image data to determine a test result, and reportthe test result to the user. The assay development time can be unique toeach test. In some embodiments walkaway mode can be set bydouble-clicking the single button 175 of the device 160. Further inputcan indicate the assay development time to the reader device. Forexample, a barcode scanned by a barcode reader of the device 160, or abarcode provided on the assay or on a cartridge used to hold the assay,can indicate to the device 160 a type of assay that is inserted and adevelopment time for that assay. Based upon the type of assay, thedevice 160 can wait for the predetermined amount of time after sampleapplication and insertion before collecting image data representing theassay.

There are many advantages associated with the ability of a user toselect and switch between device operation modes in implementations ofbase assay analyzers described herein. The endpoint read mode can beconvenient in large laboratories or medical practice facilities wherepersonnel typically batch process a number of tests. The walkaway modecan be useful when a single test is being performed, or when the enduser does not want to have to track the assay development time (or isnot knowledgeable or not trained on how to track the assay developmenttime accurately). The walkaway mode can advantageously reduce oreliminate the occurrence of incorrect test results due to an assay beinginserted and read (for example, imaged) too quickly (too soon before thedevelopment time of the assay has elapsed) or too slowly (too long afterthe development time of the assay has elapsed). Further, in walkawaymode the assay reader can operate to capture multiple images of theassay at predetermined time intervals, for example when a kinetic graphof the assay readings is desired.

One embodiment of the disclosed device 160 includes only a single button175 on its exterior housing, such as a single power button that powersthe device 160 off and on. Embodiments of the disclosed device 160 alsoimplement two different device operation modes (although more than twodevice operation modes are possible). In order to enable the end user toselect and switch between the two device operation modes, the device 160can include instructions to implement a double-click function on thepower button. After receiving input of a single press of the button topower on the device, insertion of an assay cartridge can automaticallytrigger end-point read mode. When the processor of the device receivesinput from a user double-clicking the power button, this can initiatethe stored instructions to implement the walkaway mode. Thisdouble-click functionality offers a simple and intuitive way for the enduser to switch between different operation modes of the base assayanalyzer. The double-click functionality also enables the user toconfigure the device in real time to operate in the walkaway modewithout requiring any additional configuration steps or additionalprogramming of the device 160 by the user. It will be appreciated thatthe device 160 can be provided with instructions to recognize otherclick modes instead of or in addition to the double-click to triggersecondary (non-default) device operation modes, for example to recognizea user pressing the button any predetermined number of times, pressingthe button in a predetermined pattern, and/or pressing and holding thebutton for a predetermined length of time.

As described above, the device 160 can also include a display 180 fordisplaying instructions and/or test results to the user. After insertionof the test strip, the device 160 can read a bar code on the assay teststrip to identify the name, permissible concentration ranges of thedrug, and/or maximum permissible concentration of the drug. The device160 can image the inserted test strip, and analyze the signalsrepresenting the imaged test strip to calculate results, display theresults to the user, and optionally transmit and/or locally store theresults. The results can be calculated and displayed as contaminationwith an indication of positive or negative (for example, +/−; yes/no;etc.), and/or the actual amount of contamination (analyte of interest)per area (for example, Drug Concentration=0.1 ng/cm²) and/or actualamount of contamination (analyte of interest) per volume of buffersolution (for example, Drug Concentration=3 ng/ml). These indicationsare non-limiting examples as other indications and measurement units arealso suitable.

Some embodiments of the device 160 may simply display the result(s) tothe user. Some embodiments of the device 160 may also store theresult(s) in an internal memory that can be recalled, for example, byUSB connection, network connection (wired or wireless), cell phoneconnection, near field communication, Bluetooth connection, and thelike. The result(s) can also automatically be logged into the facilityrecords and tracking system of the environment (for example, facility)where the test is performed. The device 160 can also be programmed toautomatically alert any additional personnel as required, withoutfurther input or instruction by the user. For example, if the device 160reads contamination levels that are above the threshold of human uptakeand considered hazardous to for human contact, a head pharmacist, nurse,manager, or safety officer can be automatically notified with theresults and concentration of contamination to facilitate a rapidresponse. The notification can include location information, such as butnot limited to a geographic position (latitude/longitude) or descriptionof location (Hospital A, Patient Room B, etc.). That response mayinclude a detailed decontamination routine by trained personnel or usinga decontamination kit provided together or separately from the hazardouscontamination detection kit.

In some embodiments, device 160 can be a special-purpose assay readerdevice configured with computer-executable instructions for identifyingtrace concentrations of contaminants in the samples applied to teststrips. In other embodiments other suitable liquid sample test systemscan be used to identify the presence and/or concentration of a hazardousdrug.

Overview of Example Devices and Techniques for Augmented Reality AreaSampling

FIG. 2 depicts an example augmented reality display 200 of a test areasampling environment 210 as described herein, which can be displayed forexample at block 115 of the process 100 described above. The samplingenvironment 210 includes a surface 225 identified for hazardouscontamination sampling. The surface 225 may be suspected of havinghazardous contamination or known to have hazardous contamination. Insome cases, the surface 225 is suspected of not having hazardouscontamination but is tested periodically, for example as part of aroutine maintenance program, to confirm there is in fact no hazardouscontamination. In some examples, a user tests the surface 225 based on apre-established routine maintenance schedule, such as on the half hour,hourly, daily, weekly, monthly, or some other periodicity.

Surface 225 can be in a pharmacy where hazardous drugs are handled ordispensed, in an environment used for treatment of patients withhazardous drugs, or an environment used for storage, testing, ormanufacturing of hazardous drugs, to name a few non-limiting examples.For example, surface 225 can be a biological safety cabinets andisolators (“glove box”), countertops of varying materials and locations,floors, IV poles, and administration areas (e.g., chairs, desktops,keyboards, computer screens). Other examples of surface 225 includelocations of drug transport such as shipping containers, carts, andstorage areas (e.g., shelving and refrigerators). It will be understoodthat implementations of augmented reality devices described herein canbe suitable to assist and/or instruct a user to swab any number ofsurfaces that may include a hazardous drug molecule or any other analyteof interest.

The augmented reality display 200 is illustrated as being presentedwithin the field of view of a window 205, for example of augmentedreality goggles or glasses. Other examples may have varying shapes forwindow 205 or no window at all, depending upon the type of device usedto generate and provide the augmented reality display 200.

In some implementations, the augmented reality display 200 can beprovided for an initial testing of the surface 225. In one example,testing of the surface 225 can proceed according to FIGS. 1A-1Ddescribed above. Other sampling procedures and testing devices can beused in other examples. In some implementations, the augmented realitydisplay 200 can again be displayed for follow-up testing of the surface225, for example a periodic re-check of the surface 225 or aconfirmation testing occurring after executing decontaminationprocedures to decontaminate the surface 225.

The augmented reality display 200 includes digitally-generated visualelements displayed as an overlay over the real world test area samplingenvironment 210. These include an area demarcation boundary 215,distance markings 230, and user-interface elements 220. It will beappreciated that the specific locations, shapes, and visualpresentations of these elements can vary in other embodiments whilestill providing the disclosed functionality. The example augmentedreality display 200 includes three-dimensional representations of theaugmented reality overlay elements; some or all elements can bedisplayed as two-dimensional representations in other embodiments.

The area demarcation boundary 215 denotes the specified area forsampling the surface 225 for the potential presence of a hazardouscontaminant and is accompanied by distance markings 230 to providevisual indications to the user regarding the dimensions of the areademarcation boundary 215. In some embodiments, the distance markings 230can be displayed during pre-sampling setup procedures in order to allowthe user to select a specific area for testing. In some examples, thedistance markings 230 may not be displayed during sampling.

As described herein, having a precise calculation of the sampled areacan allow a concentration of any detected contaminant per unit area ofthe sampling surface 225 to be determined with very high accuracy. Thus,in addition to displaying the area demarcation boundary 215, anaugmented reality device as described herein can also monitor samplecollection processes to perform one or more of the following: (i)identify a percentage of the area actually sampled, (ii) identify anyadditional area outside of the area demarcation boundary 215 that wassampled, (iii) compute total actual sampled area, and (iv) provide anindication when the total area has been sampled.

The example user-interface elements 220 include a shape selection buttonand a location adjustment button. The shape selection button can allowthe user to select a shape and/or size for the test area demarcationboundary 215. For example, the user can “touch” the user-interfaceelements 220 by placing a hand or finger on or within the illustrated 3Dvolume to select features of the test area demarcation boundary 215?. Inother implementations the test area shape and size can be predefined andthe shape selection button can be omitted. The location adjustmentbutton can allow the user to move the position of the test areademarcation boundary 215 in at least one direction across the surface225. In some embodiments, the device used to display the augmentedreality display 200 can analyze an image of the test area samplingenvironment 210 and automatically identify a height and/or contour ofthe surface 225, and can overlay the test area demarcation boundary 215onto the determined height and/or contours of the surface 225. Otherexamples can have varying buttons providing various user-input featuresas required for system operation and sample acquisition procedures.

FIG. 3 depicts a high level schematic block diagram of an exampleaugmented reality device 300 that generates and displays the exampledisplay of FIG. 2. The device 300 includes a number of differentcomponents for generating and presenting augmented reality views to auser, for example image capture device 330, display 315, processor(s)325, connectivity device 310, user interface controls 305, positionsensor(s) 320, a working memory 345, and a number of data repositories.The data repositories include boundary data repository 335, swabbed areadata repository 340, and test data repository 390. Though shownseparately in FIG. 3 for purposes of clarity in the discussion below, itwill be appreciated that some or all of the data repositories can bestored together in a single memory or set of memories. The workingmemory 345 stores a number of processing modules including overlaymodule 350, UI command handler 355, position tracker 360, gesturerecognition module 365, area calculator 370, communication handler 375,identification module 380, and results calculator 385. Each module canrepresent a set of computer-readable instructions, stored in a memory,and one or more processors configured by the instructions for performingthe features described below together.

The device 300 can be any device suitable for a creating a visualexperience that blends digital content (e.g., the example augmentedreality overlay of FIG. 2) with the physical world (e.g., the testenvironment) into a composite scene. For example, device 300 can be awearable device configured to display the augmented reality overlayingthe test environment to one or both eyes of a user. Device 300 can beimplemented as a heads up display, augmented or virtual reality goggles,smart glasses, or any suitable immersive or see-through augmentedreality system. Immersive displays block a user's view of the realworld, for example presenting an image of the real world scene with adigital overlay, while see-through systems leave the user's view of thereal world open and display an image overlaying the view.

Image capture device 330 acquires images of the test environment. Insome embodiments, these images can be displayed to the user with anaugmented reality overlay as described herein. In other embodiments, theimages can be used by the processor(s) 325 to generate and/or maintainpositioning of the augmented reality overlay (e.g., user interface) forexample using a transparent display, though the images themselves maynot be displayed to the user. The device 300 can also use imagescaptured by the image capture device 330 to determine informationrelating to a test, or to determine test results. For example, thedevice 300 can identify in a captured image any number of markings on aswab or test strip, for example a unique identifier (e.g., 1D or 2Dbarcode, QR code, serial number etc.) that identifies the swab and/ortest strip used. The image capture device 330 can also be used to recordthe location where the sample is taken (e.g., visual cues or identifierson a pharmacy hood or on the test surface). Using image data from theimage capture device 330, the device 300 can create a tracking record ofall items used during sample collection and test result calculation, allareas collected from, and can use this data to generate properdocumentation of the testing events and results.

The image capture device 330 can comprise, in various embodiments, acharge-coupled device (CCD), complementary metal oxide semiconductorsensor (CMOS), or any other image sensing device that receives light andgenerates image data in response to the received image. A sensor of theimage capture device 330 can have an array of a plurality ofphotosensitive elements. The photosensitive elements can be, forexample, photodiodes formed in a semiconductor substrate, for example ina CMOS image sensor. A number of pixels in captured images cancorrespond to the number of photosensitive elements in some embodiments.

The display 315 of the device 300 can present a composite scene of atest environment and augmented reality overlay to a user. The display315 can be a variety of display panels (e.g., LED, LCD, OLED panels) oroptical materials (e.g., transparent glass and/or plastic lenses orpanels) as described below. In some implementations the display 315 maybe a near-eye wearable display. In some implementations display 315 canbe a stereoscopic display or displays by which each eye is presentedwith a slightly different field of view so as to create a 3D perceptionof the composite scene.

In some implementations, the display 315 may be transparent ortranslucent so that the user can see the test environment through thedisplay 315, with the display 315 used to present the augmented realityoverlay. In such embodiments, the augmented reality overlay can beprojected onto the display 315 by a projection device positioned to emitlight into or onto the display 315, or the augmented reality overlay canbe presented by changing visual appearance of pixels of the display 315.Thus, the display 315 may be incorporated into the transparent lens(es)of a pair of goggles or glasses or of a heads-up display panel.

With a see-through (e.g., transparent or translucent) near-eye opticalsystem, the augmented reality overlay may not be displayed in-focus onthe display surface. Within a certain close range of distances from auser's eye, displaying the overlay in-focus on a semi-transparentsurface may not create an effective composite scene, as the human eyecannot comfortably focus on something too close (e.g., within 6.5 cm fora typical human eye). Thus, rather than presenting the overlay on thesurface, the display 315 can include an optical system configured toform an optical pupil and the user's eye can act as the last element inthe optical chain, thereby creating the in-focus image of the overlay onthe eye's retina. For example, a see-through near-eye display caninclude an illumination source configured to emit light representing theaugmented reality overlay and a waveguide optical element which collectsthe light and relays it towards the user's eye. Such an arrangement canform the optical pupil and the user's eye acts as the last element inthe optical chain, converting the light from the pupil into an image onthe retina. This structure can allow for non-transparent portions of thedisplay 315 to be positioned so as to not obstruct the user's view, forexample on the side of the head, leaving only a relatively smalltransparent waveguide optical element in front of the eye.

Other embodiments of the display 315 can include an opaque displaypanel, for example incorporated into an augmented or virtual realityheadset. An opaque display 315 may alternatively be incorporated intoanother computing device in some implementations, for example a user'ssmartphone or another wearable-sized computing device. As such, in someembodiments the device 300 can include a wearable structure for holdingthe display 315 of the computing device 300 in the field of view of theuser. The various modules and memory components illustrated in FIG. 3can be incorporated into the computing device and/or a samplingapplication adapted to run on the computing device, into the wearablestructure, or split between the two in various embodiments.

Some embodiments of device 300 can be a virtual retinal display devicethat projects augmented reality overlays directly onto the retina of auser's eye. Such devices can include a projection device, for example alight source and one or more lenses, in place of a display panel.

Device 300 can include one or more position sensors 320. For example, aposition sensor can be an accelerometer or gyroscope that may be used todetect in real time the viewing angle or gaze direction of the user.This data can be used to position or re-position the overlay relative tothe real-world test environment so that displayed features, for examplethe boundary of the test area, appear to maintain static positioningrelative to the test environment. To illustrate, the user may set theboundaries of the test area before swabbing, and then may turn her headduring swabbing of the test area to track the motion of the swab. Thedevice 300 can track the gaze direction of the user and can use thisdirection information to keep the positioning of the visualrepresentation of the test area boundary consistent and stationaryrelative to the test surface, even while the user turns her head. Asthis adjustment is carried out in real time, an illusion of theaugmented reality overlay merging with physical elements of the realworld may be achieved.

Connectivity device 310 can include electronic components for wiredand/or wireless communications with other devices. For example,connectivity device 310 can include a wireless connection such as acellular modem, satellite connection, or Wi-Fi, or via a wiredconnection. Thus, with connectivity device 310 the device 300 can sendor upload data to a remote repository via a network and/or receivingdata from the remote repository. As such, the data relating to test areaswabbing generated by device 300 (for example but not limited to testarea boundary size and actual area sampled), can be provided to remotedata repositories, for example in test devices used to analyze thecollected samples. A module having a cellular or satellite modemprovides a built-in mechanism for accessing publicly available networks,such as telephone or cellular networks, to enable direct communicationby the device 300 with network elements or testing devices to enableelectronic data transmission, storage, analysis and/or dissemination. Insome implementations this can be performed without requiring separateintervention or action by the user of the device, for example upondetecting completion of sampling (e.g., identifying via automated imageanalysis that the user has inserted the swab into a container and thuscompleted sampling). In some embodiments connectivity device 310 canprovide connection to a cloud database, for example a server-based datastore. Such cloud based connectivity can enable ubiquitous connectivityof a network of augmented reality test devices without the need for alocalized network infrastructure. Further, in some examples connectivitydevice 310 can enable wireless transmission of software updates to thedevice 300 (and to similar devices within a designated environment orgroup of users), for example relating to updates to size and/or locationof test areas within a clinical environment, updated test analysisalgorithms, updated threshold concentration levels, software fixes, andthe like.

Device 300 can include UI controls 305, for example mechanical buttons,touch-sensitive buttons, a touch-sensitive panel, joysticks, inputwheels, and the like for receiving input from a user regarding operationof the device 300. Some implementations can additionally oralternatively receive user input by analyzing images from the imagecapture device 330, for example to identify known command gestures,interaction with elements displayed in the augmented reality overlay(e.g., user-interface elements 220), and/or to track the position of auser's hand and/or a swab during sampling.

Processor(s) 325 include one or more hardware processors configured toperform various processing operations on received image data forgenerating and displaying augmented reality overlays, defining testareas, and tracking sampled areas, for example. Processor(s) 325 caninclude one or more of a dedicated image signal processor, a graphicsprocessing unit (GPU), a general purpose processor, a digital signalprocessor (DSP), an application specific integrated circuit (ASIC), afield programmable gate array (FPGA) or other programmable logic device,discrete gate or transistor logic, discrete hardware components, or anycombination thereof designed to perform the functions described herein.

As shown, processor(s) 325 are connected to a working memory 345 storinga number of modules. As described in more detail below, these modulesinclude instructions that configure the processor(s) 325 to performvarious image processing and device management tasks. Working memory 345may be used by processor(s) 325 to store a working set of processorinstructions contained in the modules of memory 345. Working memory 345may also be used by processor(s) 325 to store dynamic data createdduring the operation of device 300. In some implementations, a designmay utilize ROM or static RAM memory for the storage of processorinstructions implementing the modules contained in memory 345. Theprocessor instructions may be loaded into RAM to facilitate execution bythe processor(s) 325. For example, working memory 345 may comprise RAMmemory, with instructions loaded into working memory 345 beforeexecution by the processor(s) 325.

Boundary data repository 335 is a data storage device that stores datarepresenting size and location of a test area boundary. For example,boundary data repository 335 can store dimensions (e.g., width andlength) of a test area, and can further store information regardingpositioning of the test area boundary relative to one or both of thedevice 300 and automatically-identified features in image datarepresenting the test area. Thus, the boundary data repository 335 canstore information regarding size and location of the test area boundarywithin a three-dimensional coordinate frame around the device 300. Insome implementations, boundary data repository 335 can store a number ofoptions regarding test area boundaries (e.g., different sizes) and theseoptions can be made available for selection by the user at the beginningof setup for contaminant sampling. In some implementations, the device300 can automatically select a test area boundary size for a particularsampling process, for example using information identifying one or moreof the test area, a sample collection kit being used for the test areasampling, and a test device that will be used to test the sample. Insome implementations, the data in the boundary data repository can beinput by a user, either manually via user input controls or via adetected gesture input, for example by the user drawing a boundary overthe test area with a hand.

Swabbed area data repository 340 is a data storage device that storesdata representing the actual area swabbed during a hazardous contaminantsampling procedure. The swabbed area data repository 340 can be updatedduring the course of a sampling procedure to reflect the unit area(e.g., cm²) and/or percentage (of demarcated or non-demarcated) testarea that has been swabbed by a user. This data can be determined by thearea calculator module 370 as described in more detail below.

Test data repository 390 is a data storage device that storesinformation relating to the sampling procedure. This data can includeidentifiers representing an operator performing the procedure, thelocation of the test area, a sampling kit or device used to collect thesample from the test area, a test device to analyze the collectedsample, and the specific antineoplastic drug or other contaminant soughtto be detected by the testing, to name a few non-limiting examples. Thedata in test data repository 390 can include parameters of thecollection and/or test devices in some implementations, for exampleparameters relating to area sampling such as swab size. The test datarepository 390 can also include specific personnel associated with asampling procedure as well as contact information for such personnel.

In some implementations, the test data repository 390 can be used tostore and analyze aggregate test data from a specific location, by aspecific user, or using a particular type of collection/test device at anumber of different points in time. The test data repository 390 canalso be used to store aggregate test data from a number of differenttest environments or sampling locations. Thus in some embodiments thetest data repository 390 may be stored on, or mirrored to, a remote datarepository, for example a repository in network communication with anetwork of different augmented reality devices and test devices.Beneficially, this can increase traceability of the sampling proceduresperformed by storing information on devices used for tests, areassampled, results of sample analysis, and associated documentationregarding test operators.

Identification module 380 is a module configured to identify datarelating to the sampling procedure, for example the types of datadescribed as stored in the test data repository 390 as described above.The identification module 380 can be configured to receive informationregarding a scanned or imaged bar code, serial number, or otheridentifier and identify a corresponding user, test area, collectiondevice, test device, and the like. For example, locations identified forsampling (e.g., a pharmacy hood or counter) can be pre-marked with areflector array, bar code, or other identifier that would help thedevice 300 identify the test area as a pre-identified specific location.In some embodiments, information generated by the identification module380 can be used to select or recommend a test area size for a particularsampling procedure.

UI command handler 355 is a module configured to manage systemoperations in response to user commands. For example, UI command handler355 can store a test area boundary size in the boundary data repository335 in response to user drawing, selection, or other input commandsdesignating the size and/or location of the test area boundary. UIcommand handler 355 can also cause storage and/or transmission of testdata (e.g., actual sampled area and other information stored in the datarepositories 335, 340, 390) to remote devices (e.g., a database of ahealthcare organization, a test device) in response to user commands.

Overlay module 350 is a module configured to generate, update, and causedisplay of augmented reality overlays. As described herein, an overlaycan include a visual representation of a test area and/or test areaboundary displayed over the test surface in order to guide a user insampling a specific area. An overlay can also include modifications tothe visual representation of the test area to indicate areas that havealready been swabbed (e.g., change in color, brightness, or patternoverlaying the test area or even areas outside the test area that wereswabbed unintentionally). Some embodiments can display a trail or trackwhere swabbing has occurred. An overlay can further include various userinterface elements in some implementations. In some embodiments, theoverlay can include visually-displayed instructions to guide the userthrough the various steps of the sampling process. In some cases,audible instructions are provided to the user. The sizes, locations, andorientations of elements of an augmented reality overlay may be fixedrelative to the three-dimensional coordinate frame around the device300, and during display of the augmented reality overlay the elementscan be positioned according to their fixed sizes, locations, andorientations within the coordinate frame even as the field of view ofthe device 300 changes.

Position tracker 360 is a module configured to track location of thetest area throughout a sampling procedure. Initially, the positiontracker 360 can be used to establish a position for the test area (e.g.,its size and location) relative to the test environment and/or device.As described above, the test area can be mapped to the three-dimensionalcoordinate frame surrounding device 300. In some embodiments, theposition tracker 360 can set the test area location relative to featuresidentified in images of the test environment (e.g., the test surface).In some embodiments, the position tracker 360 can set the test arealocation relative to the initial positioning of the device 300 asdetermined by the position sensor(s) 320. Position tracker 360 storesthis data in the boundary area data repository 335, and can receive datafrom the image capture device 330 and/or position sensors 320 in orderto track the location of the test area relative to one or both of thedevice 300 (e.g., by using data from position sensors 320 to identifymovement of the device) and the real-world test environment (e.g.,through automated image analysis). The position tracker 360 canadditionally track the movement of the device 300 and/or the field ofview of the image capture device 330 through the coordinate frame, andcan use this information in combination with the stored sizes,locations, and orientations of overlay elements in order to determinehow to position specific overlay elements within the area of theoverlay. Position tracker 360 can thus cooperate with the overlay module350 to maintain a consistent location of the visual representation ofthe test area boundary in overlays presented to the user. For example,as the user moves throughout the test environment, the position tracker360 can send updates to the overlay module 350 regarding where toposition the test area depiction in the overlay, so that the depictionof the test area can be displayed in the same position relative to thereal-world test environment even as the user moves.

Gesture recognition module 365 is a module configured to identifygestures made by a swab, or by the hands and/or fingers of a user. Suchgestures can include, for example, command gestures (e.g., initiate swabtracking, swabbing complete), swabbing motions (e.g., for trackingactual swabbed area), and press, select, drag, and/or swipe gestures forinteracting with buttons or other augmented reality overlay userinterface features. In some embodiments, the device 300 may be providedtogether with one or more trackers that the user can wear on fingers orhands, or secure to a sampling swab handle, to facilitate gesturerecognition and sampled area tracking. Such trackers can includeaccelerometers, gyroscopes, electromagnetic (EM) position sensorspassing through an EM field generated around the test environment, andother suitable position sensors, and/or can include optical markers(e.g., specifically-colored materials or reflective materials).Similarly, a swab provided for use with the device 300 can haveprocessing intelligence that communicates with the device 300, forexample by having a tracking beacon (e.g., reflector array, active radiofrequency, position sensor, etc.) that can be used during sampling toenable a more accurate assessment of actual swabbed area. Positionsensors can communicate with the device 300 via the connectivity device310 in some implementations. In the case of optical markers, the gesturerecognition module can include instructions to identify and track thelocation of such markers in data received from the image capture device330. In some embodiments, the boundary of a sample collection swab canbe marked with optical markers in order to facilitate determination bythe device 300 of actual area of the test surface that passes underneaththe swab material. Gesture recognition module 365 can identify pixels incaptured images that correspond to the test area, can identify and log(in swabbed area data repository 340) one or both of pixels thatcorrespond to locations that have been swabbed and pixels thatcorrespond to locations that have not been swabbed, and can determinewhen the number and locations of logged swabbed and/or unswabbed pixelsindicate that the entire test area has been swabbed.

Area calculator 370 is a module configured to calculate the actual areaswabbed during a sampling procedure. Area calculator 370 can receive oneor more of the following: (i) data from the boundary data repository 335regarding a set size and location of the test area within thethree-dimensional coordinate frame set by device 300, (ii) data from theoverlay module 350 regarding a current position of the test area in anaugmented reality overlay and/or field of view 395 of image capturedevice 330, (iii) data from the gesture recognition module 365 regardingmovement of the swab and/or a user's hand through the test area duringsample collection, and (iv) data from the test data repository 390regarding swab size. Area calculator 370 can use the received data tocalculate the actual area that has been swabbed during sample collection(both within and outside of the designated test area boundary) and/orpercentage of the test area that has been swabbed. In some examples, theamount of swabbed area outside of the test area boundary can be used toadjust the confidence level of the test result (the presence and/orconcentration of the contaminant of interest).

In one example, the area calculator 370 can receive data from theswabbed area data repository 340 identifying logged pixels from aplurality of images that are determined to have been swabbed by theuser, can use a mapping between the scene depicted in each image and thethree-dimensional coordinate frame to determine a two-dimensional areaof the test surface represented by the logged swabbed pixels, and canuse distance measurements within the three-dimensional coordinate frameto determine the swabbed area represented by the two-dimensional area ofthe test surface. Though described in the context of an example flattest surface, such area calculations can also factor in any identifiedthree-dimensional contours of the test surface.

Some embodiments of the area calculator 370 can compare the swabbed areato a threshold or predetermined minimum area, and the device 300 canalert a user when an area greater than or equal to the predeterminedminimum area has been swabbed. As such, some embodiments of the device300 may not require marking of a specific area, but rather can keep arunning tally of total swabbed area for comparison to the predeterminedminimum area.

Optionally, some embodiments of the device 300 can include the resultscalculator 385. Results calculator 385 is a module configured todetermine the presence and/or concentration of a hazardous drug in aliquid sample, for example a sample collected using the guidanceprovided by the device 300. For example, the results calculator 385 canreceive image data representing an image depicting a test device fromthe image capture device 330. In one example, the image can depict thedisplay of such a test device, with the display providing an indicationof the presence and/or concentration of a hazardous drug. The resultscalculator 385 can identify the test results indicated in the image ofthe display. In another example, the results calculator 385 can receiveimage data representing an image depicting a test strip, for example alateral flow assay test strip including one or more test areas and oneor more control areas. In such examples, the results calculator 385 canidentify the saturation level (or other optically-detectable change) ofany test and control areas of the test strip based on color and/orintensity values of pixels corresponding to the locations of the lineson the test strip, and can use the identified saturation level todetermine the presence and/or concentration of the hazardous drug basedon the identified saturation level(s). For example, in a competitivelateral flow assay a test area can be configured to produce fullsaturation (color intensity) with no sample, and a sample with a rangeof antineoplastic drug concentrations will yield less than a maximumsaturation. The test areas of non-competitive assays can produce nosaturation with no sample, and a sample with a range of antineoplasticdrug concentrations will yield a range of saturations up to aconcentration that corresponds to full saturation.

In order to associate specific pixels in the image of the test stripwith the locations of one or more lines on the test strip and toassociate specific saturation levels with specific concentration levels,the device 300 can access test strip configuration data in the test datarepository 390 (for example as identified from an imaged barcode on atest strip cartridge). In some examples, an augmented reality overlay onthe display 315 of the device 300 can present an outline on the display315 showing a desired placement for the test strip during test stripimaging in order to aid in identifying the locations of the one or morelines. The device 300 can monitor captured images received from imagecapture device 330, can determine when the test strip has been placedaccording to the outline, and can analyze an image of the test strip inthe desired placement to determine saturation levels.

Communication handler 375 is a module configured to manage communicationfrom device 300 to external devices using the connectivity device 310.For example, communication handler 375 can be configured to transmittest data (e.g., actual sampled area and other information stored in thedata repositories 335, 340, 390) to remote devices (e.g., a database ofa healthcare organization, a test device used to analyze the sample) inresponse to commands identified by the UI command handler 355. In someembodiments, such data can be sent automatically without requiringfurther input from the user upon the occurrence of a specific event, forexample completion of sampling. Device 300 can programmatically identifycompletion of sampling in a number of different ways including anexplicit indication by the user (e.g., selection of a “samplingcompleted” UI element), implicit indications by the user (e.g., leavingthe test environment, inserting the swab into a collection container),or a predetermined period of time after initiation of the device 300 forguidance of area sampling.

Communication handler 375 can also handle transmission of any alerts topersonnel associated with a sampling procedure, for example alerts thatsampling has been completed and/or that the test area was sampledaccording to pre-specified performance standards. In some embodimentsthe device 300 may determine the results of testing the collected sampleand can additionally or alternatively provide alerts regarding anyidentified hazardous contaminant. The alerts can be provided locallywithin the test environment and/or externally to authorized personnel.For example, the augmented reality device can display a hazardindication, overlay of red or other color, or other visual indication ofcontamination over the test area. Other alert options include emittingan audible tone (e.g. a beep) or audible warning of the contamination.In some embodiments, this information can be communicated through anetwork such that any user wearing a networked augmented reality device300 who enters the test environment sees or hears the alert untilsubsequent testing indicates successful decontamination of theenvironment. Thus, some embodiments of the disclosed augmented realitydevices can form a network within a healthcare setting for providingalerts to users regarding contamination status of various environmentswithin the healthcare setting. Such networked devices can be provided tohealthcare workers, patients, visitors, and other workers within theenvironment.

FIG. 4 illustrates an example process 400 for implementing an augmentedreality test area sampling environment, for example providing thedisplay of FIG. 2 using the device 300 of FIG. 3.

The process 400 begins at block 405, in which the overlay module 350 anddisplay 315 of device 300 provide an augmented reality overlay over aview of the test environment. As described above, the view of the testenvironment can be a direct view through a transparent display or anindirect view of an image captured of the test environment.

At block 410, the position tracker 360 can set the size and/or locationof the test area boundary on a surface of the test area samplingenvironment. In some implementations the UI command handler 355 canreceive user input indicating the size and/or location of the test areaboundary set by the user. For example, the user can select the test areafrom a predetermined range of sizes or can manually input the dimensionsof the test area. In another example, the device 300 may identifythrough analysis of a series of image frames (e.g., a video) that theuser draws the test area over the test surface. In some examples, thedevice 300 can automatically identify the test area size and/orposition, for example based on the type or location of the sampling.

At block 415, the overlay module 350 can add a visual representation ofthe test area and/or the test area boundary to the augmented realityoverlay. For example, the border of the test area can be displayed as atwo-dimensional rectangle or a three-dimensional box. As anotherexample, the color and/or brightness of the test area can be changed tovisually distinguish the test area from surrounding areas.

At block 420, the position tracker 360 and gesture recognition module365 can monitor user interactions with the test environment. Theseinteractions can include the user contacting the surface within the testarea with a sampling swab and moving the sampling swab across thesurface. Block 420 can include monitoring a position of the swab withinthe test area (and optionally identifying swabbing outside of the testarea) and in some implementations can further include confirming thatthe swab is in contact with the test surface. At block 420, the device300 can also provide a notification to the user when he swabs outside ofthe test area too often or too much.

At decision block 425, the device 300 can determine whether the entiretest area has been swabbed. If not, the process 400 loops back to block420 to monitor user interactions with the test environment. The devicecan visually indicate to the user what areas have not been swabbed, forexample by overlaying the unswabbed area with a color, pattern, texture,etc.

If the device 300 determines at block 425 that the entire test are hasbeen swabbed, some implementations can transition automatically to block430. Other implementations can transition to block 430 after receiving auser input that sampling is completed or by programmatically identifyingcompletion of sampling. At block 430, the device 300 can calculate,store, and/or transmit the sampled area. For example, the areacalculator 370 can generate a final calculation of the actual areasampled by the user during the process 400. This calculation can bestored in the swabbed area data repository 340 in association with thesampling procedure in some embodiments. In other embodiments, thecommunication handler 375 can cause transmission of the final calculatedarea and any other specified information relating to the test to aremote device, for example a test device designated for analyzing theliquid sample and/or healthcare facility database.

Thus, by using device 300 and process 400, a user can be confident thatthe proper area has been swabbed and/or that the test device has preciseinformation regarding the swabbed area. Beneficially, this enables moreaccurate determinations (by device 300 or another testing device)regarding the concentration of any detected contaminant on the testsurface.

FIGS. 2-4 discussed above represent one embodiment for accuratelytracking sampled area during surface contamination testing using awearable augmented reality display device 300. FIGS. 5A-7, discussedbelow, represent another embodiment for accurately tracking sampled areausing an augmented reality projection device 600.

FIGS. 5A and 5B depict an example augmented reality projection onto atest area sampling environment as described herein, for example at block115 of the process 100 described above or using process 700 describedbelow. FIG. 5A depicts an initial configuration 500A of the augmentedreality projection and FIG. 5B depicts an updated configuration 500B ofthe augmented reality projection of FIG. 5A partway through areasampling. The augmented reality projections of FIGS. 5A and 5B aregenerated by a projection device 505 positioned outside of the testarea.

In the initial configuration 500A, projection device 505 projects avisual depiction of a test area 510 onto a test surface 520. Forexample, the projection device 505 projects a visual depiction of a testarea 510 via one or more light emitting devices, such as but not limitedto a laser. As shown, the test area 510 is depicted as a grid having awidth W and height H. Other visual depictions are possible within thescope of this disclosure, for example rectangular (or other shaped)boundaries without a grid or an array of dots across the test area, toname a few examples.

As shown in the updated configuration 500B in FIG. 5B, the visualdepiction of the test area 510 is changed to reflect that certainportions have already been swabbed. In the depicted example, the swabbedarea 515 at the corner of the test area 510 is no longer is overlaidwith the grid projection to represent that the user has already swabbedthis portion of the test area 510. In other examples, rather thanremoving the overlay from the swabbed area 515 the projection device 505can alter the visual representation of this area to use a differentdepiction style than used for unswabbed areas of the test area 510.Further, the device 505 can alter the projected area to include regionsof the test surface 520 outside of the demarcated area 510 that havebeen inadvertently swabbed by the user. The different depiction stylecan include, for example, a different color, a different shade of thesame color, a different pattern, a difference texture, or other visualdifference. This change is based on a dynamic measurement performed bythe device 505 as it images the test surface 520 and determines whatarea has been tested.

Beneficially, the device 505 mitigates the amount of contact between theuser and the potentially contaminated test surface, and also providesrepeatability and accuracy. For example, a technician does not have toposition and adjust the device 505 to project on this exact same areaeach time the surface is tested. In some embodiments, the device 505 canbe affixed near the test surface to provide consistency of thedemarcated area and location, as distance between the device 505 and thetest area 510 can alter the actual area within the projected demarcatedgeometry. The device 505 can be durable and can remain in use formultiple testing cycles. The consistency of the device 505 also removesrisk of user error, particularly in scenarios where the same region istested periodically, leading to strong repeatability and more reliableand accurate test results. The user may only be required to push abutton to turn on the device 505 and be presented with the demarcatedgeometry of the test area, and when testing is completed can press thebutton to switch the device off (or simply leave and wait for the deviceto enter a standby or sleep mode). This eliminates the need to removeany physical markers (e.g., sticky dots) from the test area aftertesting is completed, which is a portion of current testing processesthat potentially exposes the user to hazardous drugs.

FIG. 6 depicts a high level schematic block diagram of an exampleprojection device 600 that can be used to generate and display theexample projections of FIGS. 5A and 5B. The device 600 can be anysuitable for a projecting an image or video of a test area overlay ontoa test environment. For example, device 600 can use lasers or LEDs toproject images. The projection device 600 includes a number of differentcomponents for generating and projecting augmented reality views to auser, for example image capture device 630, projector 615, processor(s)625, connectivity device 610, a working memory 605, and a number of datarepositories. The data repositories include boundary data repository635, swabbed area data repository 640, and test data repository 620.Though shown separately in FIG. 6 for purposes of clarity in thediscussion below, it will be appreciated that some or all of the datarepositories can be stored together in a single memory or set ofmemories. The working memory 605 stores a number of processing modulesincluding projection module 645, gesture recognition module 650, areacalculator 655, and communication handler 660. Each module can representa set of computer-readable instructions, stored in a memory, and one ormore processors configured by the instructions for performing thefeatures described below together.

In some implementations, device 600 can be programmed with a specifictest area boundary size. The device 600 can be placed or affixed withina test environment so that the field of projection is directed towardthe desired sampling surface. The device 600 can be activated by a userbefore beginning contaminant sampling. In such implementations, thedevice 600 can illuminate the same area each time, which can bebeneficial for consistently testing the same area after decontaminationto assess success of the decontamination procedures, or for periodictesting to confirm the absence of contamination and/or monitor thechange in contamination level of the test environment over time. Anotherbenefit of such a device is the quick setup prior to sampling—a user cansimply activate the device to illuminate a predetermined test region andbegin sampling. Other embodiments of the device 600 can be configuredfor portability and use in a variety of sampling environments. Thedevice 600 may enable a user to select a specific test area boundary orinput test information for automatically determining the boundary. Insome examples, the device 600 is permanently or removably positioned ona stationary stand on the testing surface so that it is consistently inthe same location relative to and the same height from the samplingsurface. In another example, the device 600 is permanently orsemi-permanently affixed in a location over a benchtop, on a desk, orinside a fume hood where antineoplastic agents are handled, prepared,and/or dispensed.

Image capture device 630 is configured for acquiring images of the testenvironment. As described below, these images can be used to monitoruser interaction with the test area. The image capture device 630 cancomprise, in various embodiments, a charge-coupled device (CCD),complementary metal oxide semiconductor sensor (CMOS), or any otherimage sensing device that receives light and generates image data inresponse to the received image. In some embodiments, the image capturedevice 630 and corresponding functionality described below can beomitted, and device 600 can be used just for demarcation of test areaboundaries and not for tracking.

The device 600 can use the projector 615 to project the visualrepresentation of the test area onto the real-world test environment.Projector 615 can include at least one light source (e.g., a laser orLED light) and optionally one or more lens elements. For example, theprojector 615 of a device 600 used to present the dynamically updatingoverlay as shown in FIGS. 5A and 5B may include a laser galvanometerthat steers a laser into any desired pattern. The device 600 maydigitally control the image projected through projector 615, may useanalog components to control the image, for example transparent/coloredslides, masks, or a combination thereof.

Connectivity device 610 can include electronic components for wiredand/or wireless communications with other devices. For example,connectivity device 610 can include a wireless connection such as acellular modem, satellite connection, or Wi-Fi, or via a wiredconnection. Thus, with connectivity device 610 the device 600 can becapable of sending or uploading data to a remote repository via anetwork and/or receiving data from the remote repository. As such, thedata relating to test area swabbing generated by device 600 (for exampletest area boundary size and actual area sampled) can be provided toremote data repositories, for example in test devices used to analyzethe collected samples. A module having a cellular or satellite modemprovides a built-in mechanism for accessing publicly available networks,such as telephone or cellular networks, to enable direct communicationby the device 600 with network elements or testing devices to enableelectronic data transmission, storage, analysis and/or dissemination. Insome implementations this can be performed without requiring separateintervention or action by the user of the device, for example upondetecting completion of sampling (e.g., identifying via automated imageanalysis that the user has inserted the swab into a container and thuscompleted sampling). In some embodiments connectivity device 610 canprovide connection to a cloud database, for example a server-based datastore. Such cloud based connectivity can enable ubiquitous connectivityof augmented reality test devices without the need for a localizednetwork infrastructure. Further, in some examples connectivity device610 can enable wireless transmission of software updates to the device600 (and to similar devices within a designated environment or group ofusers), for example relating to updates to size and/or location of testareas within a clinical environment, updated test analysis algorithms,updated threshold concentration levels, software fixes, and the like.

Processor(s) 625 include one or more hardware processors configured toperform various processing operations on received image data forgenerating and projecting augmented reality overlays and trackingsampled areas, for example. Processor(s) 625 can include one or more ofa dedicated image signal processor, a graphics processing unit (GPU), ageneral purpose processor, a digital signal processor (DSP), anapplication specific integrated circuit (ASIC), a field programmablegate array (FPGA) or other programmable logic device, discrete gate ortransistor logic, discrete hardware components, or any combinationthereof designed to perform the functions described herein.

As shown, processor(s) 625 are connected to a working memory 605 storinga number of modules. As described in more detail below, these modulesinclude instructions that configure the processor(s) 625 to performvarious image processing and device management tasks. Working memory 605may be used by processor(s) 625 to store a working set of processorinstructions contained in the modules of memory 605. Working memory 605may also be used by processor(s) 625 to store dynamic data createdduring the operation of device 600. In some implementations, a designmay utilize ROM or static RAM memory for the storage of processorinstructions implementing the modules contained in memory 605. Theprocessor instructions may be loaded into RAM to facilitate execution bythe processor(s) 625. For example, working memory 605 may comprise RAMmemory, with instructions loaded into working memory 605 beforeexecution by the processor(s) 625.

Boundary data repository 635 is a data storage device that stores datarepresenting size and location of a test area boundary. For example,boundary data repository 635 can store dimensions (e.g., width andlength) of a test area, and can further store information regardingdifferent regions of a test area, for example for use in adjusting therepresentation of swabbed areas as discussed with respect to FIG. 5B.Some implementations of the boundary data repository 635 can store asingle size for a test area (e.g., one foot by one foot) and the usercan position the device 600 to project the boundary in the desiredlocation on the test surface. In some implementations, boundary datarepository 635 can store a number of options regarding test areaboundaries (e.g., different sizes) and these options can be madeavailable for selection by the user at the beginning of setup forcontaminant sampling. In some implementations, the device 600 canautomatically select a test area boundary size for a particular samplingprocess, for example using information identifying one or more of thetest area, a sample collection kit being used for the test areasampling, and a test device that will be used to test the sample. Insome implementations, the data in the boundary data repository can beinput by a user, either manually via user input controls or via adetected gesture input, for example by the user drawing a boundary overthe test area with a hand.

Swabbed area data repository 640 is a data storage device that storesdata representing the actual area swabbed during a hazardous contaminantsampling procedure. The swabbed area data repository 640 can be updatedduring the course of a sampling procedure to reflect the unit area(e.g., cm²) and/or percentage test area that has been swabbed by a user.This data can be determined by the area calculator module 655 asdescribed in more detail below.

Test data repository 620 is a data storage device that storesinformation relating to the sampling procedure. This data can includeidentifiers representing an operator performing the procedure, thelocation of the test area, a sampling kit or device used to collect thesample from the test area, a test device intended for use in analyzingthe collected sample, and the specific antineoplastic drug or othercontaminant sought to be detected by the testing, to name a fewexamples. The data in test data repository 620 can include parameters ofthe collection and/or test devices in some implementations, for exampleparameters relating to area sampling such as swab size. The test datarepository 620 can also include information about specific personnelassociated with a sampling procedure as well as contact information forsuch personnel.

In some implementations, the test data repository 620 can be used tostore and analyze aggregate test data from a specific location, by aspecific user, or using a particular type of collection/test device at anumber of different points in time. The test data repository can also beused to store aggregate test data from a number of different testenvironments or sampling locations. Thus in some embodiments the testdata repository 620 may be stored on, or mirrored to, a remote datarepository, for example a repository in network communication with anetwork of different augmented reality devices and test devices.Beneficially, this can increase traceability of the sampling proceduresperformed by storing devices used for tests, areas sampled, results ofsample analysis, and associated documentation regarding test operators.Though not illustrated, in some embodiments the device 600 can beconfigured with a test results module (similar to device 300) forreading test results from test devices, and these results can be storedin the test data repository 620.

Projection module 645 is a module configured to generate, update, andcause display of augmented reality overlays. As described herein, anoverlay can include a visual representation of a test area and/or testarea boundary displayed over the test surface in order to guide a userin sampling a specific area. An overlay can also include modificationsto the visual representation of the test area to indicate areas thathave already been swabbed (e.g., change in color, brightness, or patternoverlaying the test area). An overlay can further include various userinterface elements in some implementations.

Gesture recognition module 650 is a module configured to identifygestures made by the hands and/or fingers of a user in image datareceived from the image capture device 630. Such gestures can include,for example, command gestures (e.g., initiate swab tracking, swabbingcomplete), swabbing motions (e.g., for tracking actual swabbed area),and press, select, drag, and/or swipe gestures for interacting withbuttons or other augmented reality overlay user interface features. Insome embodiments, the device 600 may be provided with one or moretrackers that the user can wear on fingers or hands, or secure to asampling swab handle, to facilitate gesture recognition and sampled areatracking. Such trackers can include accelerometers, gyroscopes,electromagnetic (EM) position sensors passing through an EM fieldgenerated around the test environment, and other suitable positionsensors, and/or can include optical markers (e.g., specifically-coloredmaterials or reflective materials). Position sensors can communicatewith the device 600 via the connectivity device 610 in someimplementations. In the case of optical markers, the gesture recognitionmodule can include instructions to identify and track the location ofsuch markers in data received from the image capture device 630. In someembodiments, the boundary of a sample collection swab can be providedwith optical markers in order to facilitate determination by the device600 of actual area of the test surface that passes underneath the swabmaterial.

Area calculator 655 is a module configured to calculate the actual areaswabbed during a sampling procedure. Area calculator 655 can receive oneor more of the following: (i) data from the boundary area datarepository 635 regarding a set size and location of the test area, (ii)data from the gesture recognition module 650 regarding movement of theswab and/or a user's hand through the test area during samplecollection, and optionally (iii) data from the test data repository 620regarding swab size. Area calculator 655 can use the received data tocalculate the actual area that has been swabbed during sample collection(both within and outside of the designated test area boundary) and/orpercentage of the test area that has been swabbed. In some examples, thedevice 600 can provide a first audio or visual signal to the user whenthe actual swabbed area equals a minimum (or threshold) area and canprovide a second audio or visual signal (possibly different than thefirst) when the actual swabbed area equals an optimal swab area. Theuser can know after the first signal that they could stop sampling, andcan know after the second signal that they must stop sampling.

Communication handler 660 is a module configured to manage communicationfrom device 600 to external devices using the connectivity device 610.For example, communication handler 660 can be configured to transmittest data (e.g., actual sampled area and other information stored in thedata repositories 635, 640, 620) to remote devices (e.g., a database ofa healthcare organization, a test device used to analyze the sample) inresponse to commands identified by the UI command handler 355. In someembodiments, such data can be sent automatically without requiringfurther input from the user upon the occurrence of a specific event, forexample completion of sampling. Device 600 can programmatically identifycompletion of sampling in a number of different ways including anexplicit indication by the user (e.g., selection of a sampling completedUI element), implicit indications by the user (e.g., leaving the testenvironment, inserting the swab into a collection container), or apredetermined period of time after initiation of the device 600 forguidance of area sampling.

Communication handler 660 can also handle transmission of any alerts topersonnel associated with a sampling procedure, for example alerts thatsampling has been completed and/or that the test area was sampledaccording to pre-specified performance standards. In some embodimentsthe device 600 may determine the results of testing the collected sampleand can additionally or alternatively provide alerts regarding anyidentified hazardous contaminant. The alerts can be provided locallywithin the test environment and/or externally to authorized personnel.For example, the device 600 can project a hazard indication or othervisual indication of contamination onto the test area. Other alertoptions include emitting an audible tone (e.g. a beep) or audiblewarning of the contamination.

FIG. 7 illustrates an example process 700 for implementing an augmentedreality test area sampling environment, for example providing thedisplay of FIGS. 5A and 5B using the device 600 of FIG. 6.

The process 700 begins at block 705, in which the device 700 can receivea user indication to begin contaminant sample collection. In someembodiments, this indication can include the user powering on the device700. Other implementations can receive a start testing command throughuser interaction with user interface elements (projected and determinedvia image analysis or mechanically incorporated into device 700).

At block 710, the projection module 645 and projector 615 of device 600can project an augmented reality overlay onto the test environment. Thiscan include projecting an initial depiction of the unswabbed test area,for example as shown in FIG. 5A.

At block 415, the projection module 645 can add a visual representationof the test area and/or the test area boundary to the augmented realityoverlay. For example, the border of the test area can be displayed as atwo-dimensional rectangle or a three-dimensional box. As anotherexample, the color and/or brightness of the test area can be changed tovisually distinguish the test area from surrounding areas.

At block 715, the gesture recognition module 650 can monitor userinteractions with the test area. These interactions can include the usercontacting the surface within the test area with a sampling swab andmoving the sampling swab across the surface. Block 715 can includemonitoring a position of the swab within the test area (and optionallyidentifying swabbing outside of the test area) and in someimplementations can further include confirming that the swab is incontact with the test surface.

At block 720, the projection module 645 and projector 615 can update theprojection based on identified portions of the test area that have beenswabbed, for example as shown in FIG. 5B. For example, the projectionmodule 645 and projector 615 can determine not to display any overlayover the identified swabbed area of the test area to indicate that theuser has already swabbed this portion. In other examples, rather thanremoving the overlay from the swabbed area the projection device canalter the visual representation of this area to use a differentdepiction style (e.g., color, intensity, or pattern) than used forunswabbed areas of the test area.

At decision block 725, the device 600 can determine whether the entiretest area has been swabbed. If not, the process 700 loops back to block715 to monitor user interactions with the test environment.

If the device 600 determines at block 725 that the entire test are hasbeen swabbed, some implementations can transition automatically to block730. Other implementations can transition to block 730 after receiving auser input that sampling is completed or by programmatically identifyingcompletion of sampling. At block 730, the device 600 can calculate,store, and/or transmit the sampled area. For example, the areacalculator 655 can generate a final calculation of the actual areasampled by the user during the process 700. This calculation can bestored in the swabbed area data 640 in association with the samplingprocedure in some embodiments. In other embodiments, the communicationhandler 660 can cause transmission of the final calculated area and anyother specified information relating to the test to a remote device, forexample a test device designated for analyzing the liquid sample and/orhealthcare facility database.

Implementing Systems and Terminology

Implementations disclosed herein provide systems, methods and apparatusfor detection of the presence and/or quantity of hazardous drugs. Oneskilled in the art will recognize that these embodiments may beimplemented in hardware or a combination of hardware and software and/orfirmware.

The assay reading functions described herein may be stored as one ormore instructions on a processor-readable or computer-readable medium.The term “computer-readable medium” refers to any available medium thatcan be accessed by a computer or processor. By way of example, and notlimitation, such a medium may comprise RAM, ROM, EEPROM, flash memory,CD-ROM or other optical disk storage, magnetic disk storage or othermagnetic storage devices, or any other medium that can be used to storedesired program code in the form of instructions or data structures andthat can be accessed by a computer. It should be noted that acomputer-readable medium may be tangible and non-transitory. The term“computer-program product” refers to a computing device or processor incombination with code or instructions (e.g., a “program”) that may beexecuted, processed or computed by the computing device or processor. Asused herein, the term “code” may refer to software, instructions, codeor data that is/are executable by a computing device or processor.

The various illustrative logical blocks and modules described inconnection with the embodiments disclosed herein can be implemented orperformed by a machine, such as a general purpose processor, a digitalsignal processor (DSP), an application specific integrated circuit(ASIC), a field programmable gate array (FPGA) or other programmablelogic device, discrete gate or transistor logic, discrete hardwarecomponents, or any combination thereof designed to perform the functionsdescribed herein. A general purpose processor can be a microprocessor,but in the alternative, the processor can be a controller,microcontroller, or state machine, combinations of the same, or thelike. A processor can also be implemented as a combination of computingdevices, e.g., a combination of a DSP and a microprocessor, a pluralityof microprocessors, one or more microprocessors in conjunction with aDSP core, or any other such configuration. Although described hereinprimarily with respect to digital technology, a processor may alsoinclude primarily analog components. For example, any of the signalprocessing algorithms described herein may be implemented in analogcircuitry. A computing environment can include any type of computersystem, including, but not limited to, a computer system based on amicroprocessor, a mainframe computer, a digital signal processor, aportable computing device, a personal organizer, a device controller,and a computational engine within an appliance, to name a few.

The methods disclosed herein comprise one or more steps or actions forachieving the described method. The method steps and/or actions may beinterchanged with one another without departing from the scope of theclaims. In other words, unless a specific order of steps or actions isrequired for proper operation of the method that is being described, theorder and/or use of specific steps and/or actions may be modifiedwithout departing from the scope of the claims.

It should be noted that the terms “couple,” “coupling,” “coupled” orother variations of the word couple as used herein may indicate eitheran indirect connection or a direct connection. For example, if a firstcomponent is “coupled” to a second component, the first component may beeither indirectly connected to the second component or directlyconnected to the second component. As used herein, the term “plurality”denotes two or more. For example, a plurality of components indicatestwo or more components.

The term “determining” encompasses a wide variety of actions and,therefore, “determining” can include calculating, computing, processing,deriving, investigating, looking up (e.g., looking up in a table, adatabase or another data structure), ascertaining and the like. Also,“determining” can include receiving (e.g., receiving information),accessing (e.g., accessing data in a memory) and the like. Also,“determining” can include resolving, selecting, choosing, establishingand the like. The phrase “based on” can mean “based only on” and “basedat least on,” unless expressly specified otherwise.

The previous description of the disclosed implementations is provided toenable any person skilled in the art to make or use the presentdisclosure. Various modifications to these implementations will bereadily apparent to those skilled in the art, and the generic principlesdefined herein may be applied to other implementations without departingfrom the scope of the invention. Thus, the present invention is notintended to be limited to the implementations shown herein but is to beaccorded the widest scope consistent with the principles and novelfeatures disclosed herein.

What is claimed is:
 1. An augmented reality system for guidingcollection of hazardous contaminant samples, comprising: an imagecapture device configured for capturing images of a sampling environmentincluding a test surface; a display configured to display an augmentedreality overlay over a view of the sampling environment, the augmentedreality overlay including a visual representation of a boundary of atest area comprising a portion of the test surface to be swabbed tocollect a sample; at least one computer-readable memory having storedthereon executable instructions; and one or more processors incommunication with the at least one computer-readable memory andconfigured to execute the instructions to cause the system to: determinea size of the test area, determine a location of the test area relativeto one or both of the system and the sampling environment, cause outputvia the display of the visual representation of the boundary of the testarea, track movement of the swab as the user swabs the test area tocollect the sample, determine a size of an actual area swabbed by theuser based on the tracked movement, the size of the test area, and thelocation of the test area, and transmit an indication of the size of theactual area swabbed.
 2. The system of claim 1, wherein the one or moreprocessors are configured to execute the instructions to cause thesystem to analyze data from the image capture device to track movementof the swab.
 3. The system of claim 1, wherein the one or moreprocessors are configured to execute the instructions to cause thesystem to transmit the indication of the size of the actual area swabbedto a test device identified for analysis of the sample.
 4. The system ofclaim 1, wherein the one or more processors are configured to executethe instructions to cause the system to receive an indication of a levelof contamination of the test surface and output an alert to the userindicating the presence of contamination.
 5. The system of claim 1,wherein the one or more processors are configured to execute theinstructions to cause the system to receive an indication of a level ofcontamination of the test surface and output an alert to the userindicating the level of contamination.
 6. The system of claim 1, whereinthe one or more processors are configured to execute the instructions tocause the system to modify presentation of the test area in theaugmented reality overlay to show swabbed areas of the test area using afirst visual depiction and to show unswabbed areas of the test areausing a second visual depiction.
 7. The system of claim 6, wherein thefirst visual depiction is different than the second visual depiction. 8.The system of claim 6, wherein the one or more processors are configuredto execute the instructions to cause the system to identify the swabbedareas based on the tracked movement.
 9. The system of claim 6, whereinthe one or more processors are configured to execute the instructions tocause the system to display a trail over the swabbed areas.
 10. Thesystem of claim 1, wherein the one or more processors are configured toexecute the instructions to cause the system to maintain, in theaugmented reality overlay, the location of the test area relative to oneor both of the system and the sampling environment as the user movesaround the sampling environment.
 11. The system of claim 1, wherein theone or more processors are configured to execute the instructions tocause the system to: compare the actual area swabbed by the user to apredetermined threshold swabbed area, and in response to determiningthat the actual area swabbed is equal to the predetermined desiredswabbed area, provide an indication to the user to terminate swabbingthe test area.
 12. The system of claim 1, wherein the sample is a liquidsample.
 13. The system of claim 1, wherein the one or more processorsare configured to execute the instructions to cause the system to:receive data from the image capture device representing a test deviceafter provision of the sample to the test device, and analyze thereceived data to identify the presence of a hazardous contaminant and/ora level of contamination of a hazardous contaminant on the test surface.14. An augmented reality apparatus for guiding collection of hazardouscontaminant samples, comprising: an image capture device configured forcapturing images of a test surface; a projector configured to project anaugmented reality overlay onto the test surface, the augmented realityoverlay including a visual representation of a boundary of a test areacomprising a portion of the test surface to be swabbed to collect asample; at least one computer-readable memory having stored thereonexecutable instructions; and one or more processors in communicationwith the at least one computer-readable memory and configured to executethe instructions to cause the system to: cause the projector to projectthe visual representation of the boundary of the test area onto the testsurface, analyze data received from the image capture device to trackmovement of the swab as the user swabs the test area to collect thesample, determine a size of an actual area swabbed by the user based onthe tracked movement, and transmit an indication of the size of theactual area swabbed.
 15. The apparatus of claim 14, wherein the sampleis a liquid sample.
 16. The apparatus of claim 14, wherein the one ormore processors are configured to execute the instructions to cause thesystem to receive an indication of a level of contamination of the testsurface and output an alert to the user indicating the level ofcontamination.
 17. The apparatus of claim 14, wherein the one or moreprocessors are configured to execute the instructions to cause thesystem to receive an indication of a level of contamination of the testsurface and output an alert to the user indicating the presence ofcontamination.
 18. The apparatus of claim 14, wherein the one or moreprocessors are configured to execute the instructions to cause thesystem to transmit the indication of the actual area swabbed to a testdevice identified for analysis of the sample.
 19. The apparatus of claim18, wherein the one or more processors are configured to execute theinstructions to cause the system to identify the swabbed areas based onthe tracked movement.
 20. The apparatus of claim 18, wherein the one ormore processors are configured to execute the instructions to cause thesystem to modify presentation of the test area in the augmented realityoverlay to show swabbed areas of the test area using a first visualdepiction and to show unswabbed areas of the test area using a secondvisual depiction.
 21. The apparatus of claim 18, wherein the one or moreprocessors are configured to execute the instructions to cause thesystem to stop projection of a pattern onto the swabbed areas andmaintain projection of the pattern onto the unswabbed areas.
 22. Theapparatus of claim 14, wherein the one or more processors are configuredto execute the instructions to cause the system to modify presentationof the test area in the augmented reality overlay to indicate swabbedareas of the test area.
 23. The apparatus of claim 14, wherein the oneor more processors are configured to execute the instructions to causethe system to: compare the actual area swabbed by the user to apredetermined swabbed area, and in response to determining that theactual area swabbed is equal to the predetermined swabbed area, providean indication to the user to terminate swabbing the test area.
 24. Anon-transitory computer-readable medium storing instructions that, whenexecuted, cause a physical computing device to perform operations forguiding collection of a sample of a hazardous contaminant, theoperations comprising: causing output of a visual representation of aboundary of a test area for guiding a user to collect the sample of thehazardous contaminant from the test area, the test area comprising aportion of a test surface to be swabbed to collect the sample; trackingmovement of the swab as the user swabs the test area to collect thesample; determining a size of an actual area swabbed by the user basedon the tracked movement; identifying a test device designated foranalysis of the sample; and transmitting, to the test device, anindication of the size of the actual area swabbed.
 25. Thenon-transitory computer-readable medium of claim 24, wherein theoperations further comprise modifying presentation of the test area toindicate swabbed and unswabbed areas of the test area.
 26. Thenon-transitory computer-readable medium of claim 24, wherein theoperations further comprise identifying the swabbed and unswabbed areasof the test area based on the tracked movement.
 27. The non-transitorycomputer-readable medium of claim 24, wherein causing output of thevisual representation of the boundary of the test area comprisesoverlaying the visual representation over a view the test area through atransparent near eye display.
 28. The non-transitory computer-readablemedium of claim 24, wherein causing output of the visual representationof the boundary of the test area comprises: overlaying the visualrepresentation over an image of the test area to form a composite view,and displaying the composite view to the user.
 29. The non-transitorycomputer-readable medium of claim 24, wherein causing output of thevisual representation of the boundary of the test area comprisesprojecting the visual representation onto the test area.